You are here:
Quantitative variability in repeat dose toxicity studies: Implications for scientific confidence in new approach methodologies
Citation:
Paul-Friedman, K. Quantitative variability in repeat dose toxicity studies: Implications for scientific confidence in new approach methodologies. 24th Annual Green Chemistry and Engineering Conference, Virtual, NC, June 16 - 18, 2020. https://doi.org/10.23645/epacomptox.25021739
Impact/Purpose:
New approach methodologies (NAMs) for hazard are often evaluated via comparison to animal studies; however, variability in animal study data limits NAM accuracy. The US EPA Toxicity Reference Database (ToxRefDB) enables consideration of variability in effect levels, including the lowest effect level (LEL) for a treatment-related effect and the lowest observable adverse effect level (LOAEL) defined by expert review, from subacute, subchronic, chronic, multi-generation reproductive, and developmental toxicity studies. The objectives of this work were to quantify the variance within systemic LEL and LOAEL values, defined as potency values for effects in adult or parental animals only, and to estimate the upper limit of NAM prediction accuracy. Importantly, assessment of the success of NAMs that predict systemic LEL or LOAEL values should include a refined expectation of the optimal model performance to account for variance in the reference data. When evaluating predictive NAMs for systemic LELs or LOAELs, we now have the statistical information to inform understanding of their performance: for prediction of systemic LELs and LOAELs values (in ToxRefDB or similar datasets), the expectation should be that the R-squared approaches 0.5 to 0.7, and the accuracy of a predictive model should approach estimates within approximately 2 orders of magnitude on the log10(mg/kg/day) scale for excellent models, as limited by the underlying variance in systemic LEL or LOAEL values.
Description:
New approach methodologies (NAMs) for hazard are often evaluated via comparison to animal studies; however, variability in animal study data limits NAM accuracy. The US EPA Toxicity Reference Database (ToxRefDB) enables consideration of variability in effect levels, including the lowest effect level (LEL) for a treatment-related effect and the lowest observable adverse effect level (LOAEL) defined by expert review, from subacute, subchronic, chronic, multi-generation reproductive, and developmental toxicity studies. The objectives of this work were to quantify the variance within systemic LEL and LOAEL values, defined as potency values for effects in adult or parental animals only, and to estimate the upper limit of NAM prediction accuracy. Multiple linear regression (MLR) and augmented cell means (ACM) models were used to quantify the total variance, and the fraction of variance in systemic LEL and LOAEL values explained by available study descriptors (e.g., administration route, study type). The MLR approach considered each study descriptor as an independent contributor to variance, whereas the ACM approach combined categorical descriptors into cells to define replicates. Using these approaches, total variance in systemic LEL and LOAEL values (in log10-mg/kg/day units) ranged from 0.74 to 0.92. Unexplained variance in LEL and LOAEL values, approximated by the residual mean square error (MSE), ranged from 0.20-0.39. Considering subchronic, chronic, or developmental study designs separately resulted in similar values. Based on the relationship between MSE and R-squared for goodness-of-fit, the maximal R-squared may approach 55 to 73% for a NAM-based predictive model of systemic toxicity using these data as reference. The root mean square error (RMSE) ranged from 0.47 to 0.63 log10-mg/kg/day, depending on dataset and regression approach, suggesting that a two-sided minimum prediction interval for systemic effect levels may have a width of 58 to 284-fold. These findings suggest data-driven considerations for building scientific confidence in NAM-based systemic toxicity predictions. This abstract does not necessarily reflect U.S. EPA policy.
URLs/Downloads:
DOI: Quantitative variability in repeat dose toxicity studies: Implications for scientific confidence in new approach methodologies
PAULFRIEDMAN_QUANTVARINVIVO_18JUNE20_GC&E_NOAUDIO.PDF (PDF, NA pp, 895.224 KB, about PDF)